Clothing Retrieval |
Return |
Introduction
With the fast development of e-commerce and image sharing websites and the huge market of garment trade, both academic and industry have more and more laid their focus on clothing related study and applications. Many e-commerce websites, such as Taobao.com, Amazon.com, provide clothing retrieval services for users by keywords query. But keyword based retrieval can hardly meet the requirement of exact retrieval of certain style of clothing. Content based image retrieval can fill the gap to a certain extent, and there are some existing websites and applications support content based clothing retrieval. But the results of low-level feature based image retrieval still have sematic gaps with user requirement. There is a large space for improvement. We utilize the attribute based computer vision models which has been widely studied recently to augment the content based image retrieval. Relevant with clothing retrieval, clothing matching retrieval is another rising research field and industrial hotspot. Given the upper/lower body clothing images input by user, a clothing matching system outputs the lower/upper body clothing which is aesthetically and properly matched with the input clothing. There are still a limited amount of related research and applications, while the clothing matching has large actual demands and research challenges. Attribute augmented clothing retrieval and clothing matching should utilize the large amount of images and unstructured text information from online shopping websites and image sharing websites, combine theories and techniques from multiple areas, including machine learning, information retrieval, and computer vision and pattern recognition. However, the existing datasets, theories, and systems still can’t well support large scale clothing retrieval and clothing matching retrieval. To tackle the above mentioned challenges, we will conduct a thorough study on attribute augmented clothing retrieval and clothing matching retrieval. First, we will collect a large amount of clothing images, including both online shopping images and well-conducted daily street shots. Secondly, we will build a large scale annotated dataset by conducting both automatic annotation and manual annotation. Thirdly, we will design and implement attribute augmented methods for multiple retrieval scenarios. At last we will build a demo system to provide the services. We aim at providing innovative model and methods and novel techniques to contribute both the research field and future industry applications.
Framework
- Collecting clothing images from web
- Building a large scale annotated clothing image dataset
- Researching on attribute augmented clothing image retrieval and clothing matching methods
- Part and poselet based feature selection
- Within and cross scenario clothing retrieval
- Clothing matching retrieval
- Developing demo systems
As illustrated in Fig. 1, the framework of this project includes four parts: Collecting clothing images from web, building a large scale annotated clothing image dataset, researching on attribute augmented clothing image retrieval and clothing matching methods, and developing demo systems.
Collect clothing images and the text information of the images from both online shopping websites and well-edited image sharing websites. The text information includes product categories and properties provided by online shopping websites, and the description and tags of clothing images edited by image sharer. The total amount of images will be millions.
Annotate attributes, categories, and other information for the collected clothing images to form a large scale clothing image dataset. The collected text information in the previous step can be processed to conduct automatic annotation of images. Furthermore, we will build an annotation platform and recruit people to conduct manual annotation.
The clothing images are usually with noisy background and various human poses. To achieve the desired retrieval results, we will utilize the part based and poselet based approaches to select the relevant partial of features for the following attribute learning phases.
With the selected feature and annotations, we will design the clothing retrieval methods based on multiple machine learning and computer vision approaches, such as latent support vector machine, cross scenario sparse reconstruction, etc.
The well-edited street shot photos can be regarded as the knowledge pool of examples of good matching. We will design an attributed based clothing matching model, and train the model by the collected examples to learn the clothing matching fashion code automatically.
With the dataset built and the methods and models proposed, we will build a demo system to provide clothing retrieval and clothing matching services with the trained models, and get the real-time user responses for future research and improvement.